AD-A097  203 
tiwri  KCiclPfl 

,1  - 


MASSACHUSETTS  INST  OF  TECH  CAMBRIDGE  LAB  FOR  INFORMA— ETC  F/6 

res vsffi  rsss^sr^’ ,u’ 


NL 


END 


1 


iff; 


TOWARDS  FAULT-TOLERANT  OPTIMAL  CONTROL  «r 


Howard  J,  Chizeck 
Laboratory  for  Information  and 
Decision  Systems 

Massachusetts  Institute  of  Technology 
Cambridge,  Massachusetts  02139 


Ck 

APR  2 

N  A 


*  *  w* 

ECT^ 


Alan  Sv  Will  sky 

Laboratory  for  Information  and 
Decision  Systems 

Massachusetts  Institute  of  Technology 
Cambridge,  Massachusetts  02139 


to 

o 

CM 

a 


1  ABSTRACT 

Questions  regarding  the  design  of  fault-tolerant 
controllers  that  may  endow  systems  with  dynamic 
reliability  are  addressed  here.  Results  for  jump 
linear  quadratic  Gaussian  (JLQG)  control  problems 
are  extended  to  include  random  jump  costs,  tra¬ 
jectory  discontinuities,  and  a  simple  case  of  non- 
Markovian  mode  transitions.^^ 

1.  INTRODUCTION 

Regardless  of  how  well  they  are  designed  and 
manufactured,  engineering  systems  occasionally 
fail  to  function  as  expected  due  to  component  fail¬ 
ures  and  environmental  disturbances.  As  a  result, 
unacceptably  high  costs  may  be  incurred.  Ideally, 
systems  should  be  designed  to  be  dynamically  re¬ 
liable.  That  is,  they  should  function  " acceptably 
well*  despite  various  component  failures  and  envi¬ 
ronmental  disturbances.  Many  complex  engineering 
systems  currently  in  use  do  not  possess  this  pro¬ 
perty  of  f ault-tolerance .  For  example,  electrical 
power  systems  have  been  known  to  experience  com¬ 
plete  blackouts  resulting  from  the  failure  of  a 
few  components  (such  as  switches) ,  or  as  a  conse¬ 
quence  of  abrupt  disturbances  (such  as  lightning 
bolts,  and  sudden  loads). 

The  design  of  fault-tolerant  controllers  in¬ 
volves  a  number  of  subjective  questions.  Among 
these  are  modelling  issues,  particularly  with  res¬ 
pect  to  failure  events,  and  clarification  of  dif¬ 
ferent  control  tasks  such  as:  the  detection  of 
failures,  the  adaptation  and  reorganization  of 
controllers  both  in  response  to  detected  failures 
and  in  anticipation  of  them,  and  the  prevention  of 
certain  failures.  Various  costs  .  ust  be  quanti¬ 
fied  and  compared,  including  those  relating  to  ope¬ 
ration  under  different  failure  conditions,  costs 
incurred  at  failure  instants,  and  costs  related  to 
improper  failure  detection.  The  goal  is  to  find 
objective  approaches  for  the  design  of  fault- tol¬ 
erant  systems;  to  formulate  and  solve  problems 
that  capture  and  quantify  the  subjective  issues  of 
fault-tolerant  control. 

In  this  paper,  some  extensions  to  basic  re¬ 
sults  [1],(2]  concerning  the  control  of  systems 
having  randomly  jumping  parameters  are  presented, 
as  an  initial  step  towards  fault7t©iey*nt  optimal 
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2.  JUMP  LINEAR  QUADRATIC  GAUSSIAN  PROBLEM 

Our approach  is  to  model  component  failures  by 
randomly  and  abruptly  changing  parameters  (see  (3) 
for  a  survey  of  problems  of  this  type) .  Assume 
that  a  given  system  can  operate  in  N<»  different 
modes,  each  corresponding  to  a  particular  set  of 
component  and  environmental  conditions.  Motivated 
by  concerns  of  robustness  and  implementability,  as 
well  as  mathematical  tractability ,  a  linear  qua¬ 
dratic  Gaussian  problem  formulation  can  be  chosen 
for  operation  in  each  mode. 

Let  p(t)e{l, . . . ,N)  denote  the  mode  of  the 
system  at  time  t,  where  {p(t>?  t  <t£T}  is  a  finite 
state  Markov  process  having  transition  probabili¬ 
ties 

(1)  Pr{p  (t+dt)e=j  ]  p(t)=i}  sf  q^Udt+OWt)  ij*j 

U  -q^tJdt-KHdt)  i-j 

and  initial  probability  distribution  P(tQ).  The 
q^  and  q ^  are  continuous  nonnegative  functions, 
and  q  (t)  *  J  q  ,v<t) ) . 

In  between  jumps  in  p,  the  system  state  tra¬ 
jectory  x(t)  satisfies  a  vector  stochastic  dif¬ 
ferential  equation 

(2)  dx* (AXt,k)x(t)+B (t,k)u(t) )dt+C(t,k)dw(t) 
x(t  )  *  x 

°  °  n  m 

for  each  mode  k,  where  xCR  ,  ueR  is  the  control, 

w(t)  is  a  separable  Wiener  process  and  is  a 

Gaussian  random  variable  independent  of  the  dw 
increments.  Assume  that  A(t ,k) ,B(t ,k) ,C(t,k) 
piecewise-continuous  in  t  on  the  known  finite  in¬ 
terval  ft  ,T) ,  for  each  ke{lr...*N}. 
o 

Together  the  joint  process  {x(t),p(t)}  is 
assumed  to  be  Markov,  and  it  is  assumed  that  both 
.x(t)  and  p(t)  are  perfectly  observed  at  each  t. 

The  "jump  linear  quadratic  Gaussian*  control  pro¬ 
blem  involves  minimization  of  the  quadratic  cost 

functional  _ 

T 

(3)  e{j (u)  }*=E  {  f  [x<  (s)Q(s,p(s)  )x(s)  l 


■u 

|  o  +  u* 


(s) R(s,p (s) )u(s)]ds 


+  X*  (T)K^(p(T))x(T) 

where  u(t)  is  specified  by  a  feedback  control  law 
satisfying  certain  technical  conditions  (see  UJ). 
The  matrices  Q(t, j)«Q(t, j) •  >0  and  R(t,j)  ■ 

R(t,j)*  >CI  (e>0)  are  piecewise  continuous  in  t, 
(j)*0,  for  each  j.  using  either  dy- 
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namic  programming  methods  [1]  or  a  stochastic  ma¬ 
ximum  principle  [2],  it  can  be  shown  that  the 
optimal  feedback  control  law  for  operation  in  each 
mode  p(t)-j  has  the  linear  form 

(4)  u*(t)  — R‘1(t,J)B*  (t,j)K(t,j)x(t) 

where  the  synmetric  nxn  matrices  K(t,j)>0  are  spe¬ 
cified  by  N  coupled  matrix  Riccati  differential 
equations. 

This  problem  captures  some  aspects  of  fault- 
tolerant  control.  Changes  in  parameters  A,B,C 
model  abrupt  failure  events  such  as  actuator  fai¬ 
lures  ,  broken  connections,  and  the  like.  Different 
relative  weightings  can  be  assigned  to.  quantities 
such  as  performance  tolerance  and  expended  control 
energy  in  various  modes,  through  Q,  R  and  val¬ 
ues. 

When  the  mode  of  the  system  changes,  there 
may  be  random  jump  costs  incurred  that  reflect 
"start-up*  or  "shut-down"  costs,  and  transient 
costs  resulting  from  the  need  to  switch  controls. 
One  way  to  incorporate  them  in  the  optimal  control 
problem  is  to  charge  costs  x' (t)Z  (t)x(t)  when 
the  mode  shifts  from  i  to  j  at  time  t,  where  the 
2^  (t)  >0  are  independent  nxn-valued  symmetric  ma¬ 
trices  of  stochastic  processes  with  mean  value 
functions  Z^  (t)-Z^  •  (t)>0  and  finite  variances 

(and  are  independent  of  x0,pQ,w(t)).  The  cost 
functional  then  becomes 

(5)  T 

eCJ-  [u)}=e{j[u]+  \  \  /  X*  (t)  2s  .  (t)x(t)q 44  (t)dt). 
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When  the  mode  of  the  system  shifts,  there  may 
also  be  random  discontinuities  in  trajectory  x(t), 
resulting  from  impulsive  external  disturbances,  or 
phenomena  such  as  changes  in  amplifier  biases.  If 
the  modes  represent  different  linearized  models 
of  a  nonlinear  system,  jumps  in  x(t)  might  cor¬ 
respond  to  initialization  along  different  nominal 
paths.  Deterministic  discontinuities  linear  in 
x(t)  are  considered  in  {4].  Here  we  assume  that 
the  trajectory  jumps  are  described  by 

(6)  x(t+)  -  F^(t)x(0+  Hi^(t)vi^(t) 

when  the  mode  shifts  from  i  to  j  at  t.  F^jttJeR1*01 
and  are  continuous  in  t  and  deterministic; 

the  (t)  are  independent  Revalued  zero-mean 

stochastic  processes,  with  finite  variances  V,  .(t), 
independent  of  x0,pQ,w(t)  aiwl  the  Z^  (t) .  The 

cost  functional  (5),  for  a  system  described  by  (1), 
(2)  and  (6) ,  is  minimized  by  a  linear  feedback 
control  law  of  form  (4),  where  the  K(t,j)>0  are 
specified  by  the  N  coupled  equations  on  (t  ,T] : 
-K(t, j)«A(t, j)  'K(t ,  j)  +K(t,  j )  A  (t,  j)+Q(t,  j)  ° 

j)B(t,  j)X(t,  j)-^  (t)K(t,J) 

+  l  qji(t)  [Fji(t)K(t,i)Fjl(t)+2;Ji(t)J  (7) 

where  lt(I,j)»lt(j),  The  optimal  cost-to-go  from 
‘Vx(to>*p(V>  is 

(8)  x*(t  )K(t  ,p(t  ))x(t  >+r(t  ,p(t  )). where 
o  o  o  o  o  o 

the  scalar  ten  r  satisfies,  with  r(T,j)»Of 


r(t,  j)«-trlc 1  (t,j)'K(t,j)c(t,j)  J+q^ttJr  (t,  J)  (9) 

-  I  q..<t)lr(t,i)+tr{H'  <t)K(t,i)H  <t)V  (t>)j. 

31  J*  J*  J1 

The  proof  of  this  result  involves  a  straightforward 
application  of  the  Bellman  equation,  as  in  (1). 

3.  FURTHER  CONSIDERATIONS 

There  are  many  other  aspects  of  fault- tolerant 
control  that  are  not  captured  by  the  above  for¬ 
mulation.  For  example,  x(t)  and  p(t)  are  often 
not  perfectly  observable.  If  a  linear  function  of 
x(t)  is  observed  in  the  presence  of  additive 
Gaussian  white  noise  (but  p(t)  is  perfectly  ob¬ 
served)  ,  then  a  separation  (certainty  equivalence) 
result  follows,  due  to  the  linear  quadratic  for¬ 
mulation.  In  each  mode,  a  Kalman  filter  generates 
the  best  (conditional  mean)  estimate  of  x(t)  which 
is  then  used  by  the  optimal  feedback  control  law 
as  the  true  value.  If  p(t)  is  also  not  perfectly 
observed,  then  the  combined  filtering  and  control 
problem  is  much  harder  because  of  * adaptive-dual* 
difficulties;  that  is,  u(t)  can  be  used  both  to 
control  the  system,  and  to  "probe"  for  information 
useful  in  estimating  x  and  p. 

The  {p  (t) }  process  need  not  be  Harkov)  for  ex»- 
ple,  in  seme  systems  past  mode  values  and  x(t) 
histories  may  affect  mode  transition  rates.  Sup¬ 
pose  there  exists  a  stochastic  process  {0 ( t) }  such 
that  the  joint  process  (p(t),B(t>)  is  Harkov,  and 
the  intensities  in  (1)  are  of  the  form  q.(t,B(t)), 
q^  (t,£(t)) .  If  &(t)  changes  values  onlj  when 

p(t)  jumps  (and  not  in  between),  then  the  optimal 
control  law  has  the  form  (4) ,  where  the  gains 
K(t,p (t) , 3 (t) )  are  given  by  (7) -(8)  but  are  par¬ 
ameterized  by  0.  B(t)  might  correspond  to  the 
past  order  of  mode  shifts  (thus  taking  values  in  a 
finite  set)  or  to  mode  shift  times.  These  can  be 
used  to  incorporate  models  of  component  failures 
that  are  dependent  upon  elapsed  times  of  operation. 

If  B(t)  changes  values  between  p(t)  jumps  f 
the  control  problem  appears  to  be  much  more  dif¬ 
ficult.  Another  problem  formulation  (currently 
under  study)  includes  voluntary  changes  in  p(t), 
as  control  actions  with  associated  costs.  Some 
limited  results  of  this  type  are  given  in  [5] , (6) . 
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